{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "0",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "# Use AG2 in Microsoft Fabric\n",
    "\n",
    "[AG2](https://github.com/ag2ai/ag2) offers conversable LLM agents, which can be used to solve various tasks with human or automatic feedback, including tasks that require using tools via code.\n",
    "Please find documentation about this feature [here](https://docs.ag2.ai/latest/docs/user-guide/basic-concepts/conversable-agent/).\n",
    "\n",
    "[Microsoft Fabric](https://learn.microsoft.com/en-us/fabric/get-started/microsoft-fabric-overview) is an all-in-one analytics solution for enterprises that covers everything from data movement to data science, Real-Time Analytics, and business intelligence. It offers a comprehensive suite of services, including data lake, data engineering, and data integration, all in one place. Its pre-built AI models include GPT-x models such as `gpt-4o`, `gpt-4-turbo`, `gpt-4`, `gpt-4-8k`, `gpt-4-32k`, `gpt-35-turbo`, `gpt-35-turbo-16k` and `gpt-35-turbo-instruct`, etc. It's important to note that the Azure Open AI service is not supported on trial SKUs and only paid SKUs (F64 or higher, or P1 or higher) are supported.\n",
    "\n",
    "In this notebook, we demonstrate several examples:\n",
    "- 0. How to access pre-built LLM endpoints with AG2 in Microsoft Fabric.\n",
    "- 1. How to use `AssistantAgent` and `UserProxyAgent` to write code and execute the code.\n",
    "- 2. How to use `AssistantAgent` and `RetrieveUserProxyAgent` to do Retrieval Augmented Generation (RAG) for QA and Code Generation.\n",
    "- 3. How to use `MultimodalConversableAgent` to chat with images.\n",
    "\n",
    "#### Requirements\n",
    "\n",
    "AG2 requires `Python>=3.9`.\n",
    "\n",
    "Also, this notebook depends on Microsoft Fabric pre-built LLM endpoints and Fabric runtime 1.2+. Running it elsewhere may encounter errors."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "### Example 0"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "#### Work with openai<1\n",
    "\n",
    "AG2 can work with openai<1 in Microsoft Fabric. To access pre-built LLM endpoints with AG2, you can follow below example.\n",
    "\n",
    "This example can run in Fabric runtime 1.2+."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "# ag2<=0.1.14 supports openai<1\n",
    "%pip install \"ag2==0.1.14\" \"openai==0.28.1\" -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "import autogen\n",
    "\n",
    "# Set temperature, timeout and other LLM configurations\n",
    "llm_config = autogen.LLMConfig(\n",
    "    {\"model\": \"gpt-4o\"},\n",
    "    temperature=0,\n",
    "    timeout=600,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "agent = autogen.agentchat.ConversableAgent(\n",
    "    name=llm_config.config_list[0].model, llm_config=llm_config, max_consecutive_auto_reply=1, human_input_mode=\"NEVER\"\n",
    ")\n",
    "userproxy = autogen.agentchat.ConversableAgent(\n",
    "    name=\"user\",\n",
    "    max_consecutive_auto_reply=0,\n",
    "    llm_config=False,\n",
    "    default_auto_reply=\"TERMINATE\",\n",
    "    human_input_mode=\"NEVER\",\n",
    ")\n",
    "userproxy.initiate_chat(recipient=agent, message=\"Tell me a quick joke.\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "#### Work with openai>=1\n",
    "\n",
    "AG2 can work with openai>=1 in Microsoft Fabric. To access pre-built LLM endpoints with AG2, you can follow below example.\n",
    "\n",
    "This example and below examples can only run in Fabric runtime 1.3+."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# autogen>0.1.14 supports openai>=1\n",
    "%pip install \"autogen>0.2\" \"openai>1\" -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "import types\n",
    "\n",
    "from synapse.ml.fabric.credentials import get_openai_httpx_sync_client\n",
    "\n",
    "import autogen\n",
    "\n",
    "http_client = get_openai_httpx_sync_client()  # http_client is needed for openai>1\n",
    "http_client.__deepcopy__ = types.MethodType(\n",
    "    lambda self, memo: self, http_client\n",
    ")  # https://docs.ag2.ai/latest/docs/user-guide/advanced-concepts/llm-configuration-deep-dive/#adding-http-client-in-llm_config-for-proxy\\n\",\n",
    "\n",
    "# Set temperature, timeout and other LLM configurations\n",
    "llm_config = autogen.LLMConfig(\n",
    "    {\n",
    "        \"model\": \"gpt-4o\",\n",
    "        \"http_client\": http_client,\n",
    "        \"api_version\": \"2024-02-01\",\n",
    "        \"api_type\": \"azure\",\n",
    "    },\n",
    "    temperature=0,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "import autogen\n",
    "\n",
    "agent = autogen.agentchat.ConversableAgent(\n",
    "    name=llm_config.config_list[0].model, llm_config=llm_config, max_consecutive_auto_reply=1, human_input_mode=\"NEVER\"\n",
    ")\n",
    "userproxy = autogen.agentchat.ConversableAgent(\n",
    "    name=\"user\",\n",
    "    max_consecutive_auto_reply=0,\n",
    "    llm_config=False,\n",
    "    default_auto_reply=\"TERMINATE\",\n",
    "    human_input_mode=\"NEVER\",\n",
    ")\n",
    "userproxy.initiate_chat(recipient=agent, message=\"Tell me a joke about openai.\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "10",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "### Example 1\n",
    "How to use `AssistantAgent` and `UserProxyAgent` to write code and execute the code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "%pip install \"autogen[retrievechat,lmm]>=0.2.28\" -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "import types\n",
    "\n",
    "from synapse.ml.fabric.credentials import get_openai_httpx_sync_client\n",
    "\n",
    "import autogen\n",
    "\n",
    "http_client = get_openai_httpx_sync_client()  # http_client is needed for openai>1\n",
    "http_client.__deepcopy__ = types.MethodType(\n",
    "    lambda self, memo: self, http_client\n",
    ")  # https://docs.ag2.ai/latest/docs/user-guide/advanced-concepts/llm-configuration-deep-dive/#adding-http-client-in-llm_config-for-proxy\n",
    "\n",
    "\n",
    "# Set temperature, timeout and other LLM configurations\n",
    "llm_config = autogen.LLMConfig(\n",
    "    {\n",
    "        \"model\": \"gpt-4o\",\n",
    "        \"http_client\": http_client,\n",
    "        \"api_version\": \"2024-02-01\",\n",
    "        \"api_type\": \"azure\",\n",
    "    },\n",
    "    temperature=0,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "import autogen\n",
    "\n",
    "# create an AssistantAgent instance named \"assistant\"\n",
    "assistant = autogen.AssistantAgent(\n",
    "    name=\"assistant\",\n",
    "    llm_config=llm_config,\n",
    ")\n",
    "\n",
    "# create a UserProxyAgent instance named \"user_proxy\"\n",
    "user_proxy = autogen.UserProxyAgent(\n",
    "    name=\"user_proxy\",\n",
    "    human_input_mode=\"NEVER\",  # input() doesn't work, so needs to be \"NEVER\" here\n",
    "    max_consecutive_auto_reply=10,\n",
    "    is_termination_msg=lambda x: x.get(\"content\", \"\").rstrip().endswith(\"TERMINATE\"),\n",
    "    code_execution_config={\n",
    "        \"work_dir\": \"coding\",\n",
    "        \"use_docker\": False,  # Please set use_docker=True if docker is available to run the generated code. Using docker is safer than running the generated code directly.\n",
    "    },\n",
    "    llm_config=llm_config,\n",
    "    system_message=\"\"\"Reply TERMINATE if the task has been solved at full satisfaction.\n",
    "Otherwise, reply CONTINUE, or the reason why the task is not solved yet.\"\"\",\n",
    ")\n",
    "\n",
    "# the assistant receives a message from the user, which contains the task description\n",
    "chat_result = user_proxy.initiate_chat(\n",
    "    assistant,\n",
    "    message=\"\"\"\n",
    "Who should read this paper: https://arxiv.org/abs/2308.08155\n",
    "\"\"\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "print(f\"Cost for the chat:\\n{chat_result.cost}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "15",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "### Example 2\n",
    "How to use `AssistantAgent` and `RetrieveUserProxyAgent` to do Retrieval Augmented Generation (RAG) for QA and Code Generation.\n",
    "\n",
    "Check out this [blog](https://docs.ag2.ai/latest/docs/blog/2023/10/18/RetrieveChat) for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "import tempfile\n",
    "\n",
    "from autogen.coding import LocalCommandLineCodeExecutor\n",
    "\n",
    "# Create a temporary directory to store the code files.\n",
    "temp_dir = tempfile.TemporaryDirectory()\n",
    "\n",
    "# Create a local command line code executor.\n",
    "code_executor = LocalCommandLineCodeExecutor(\n",
    "    timeout=40,  # Timeout for each code execution in seconds.\n",
    "    work_dir=temp_dir.name,  # Use the temporary directory to store the code files.\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "from autogen import AssistantAgent\n",
    "from autogen.agentchat.contrib.retrieve_user_proxy_agent import RetrieveUserProxyAgent\n",
    "\n",
    "# 1. create an AssistantAgent instance named \"assistant\"\n",
    "assistant = AssistantAgent(\n",
    "    name=\"assistant\",\n",
    "    system_message=\"You are a helpful assistant.\",\n",
    "    llm_config=llm_config,\n",
    ")\n",
    "\n",
    "# 2. create the RetrieveUserProxyAgent instance named \"ragproxyagent\"\n",
    "ragproxyagent = RetrieveUserProxyAgent(\n",
    "    name=\"ragproxyagent\",\n",
    "    human_input_mode=\"NEVER\",\n",
    "    max_consecutive_auto_reply=5,\n",
    "    retrieve_config={\n",
    "        \"docs_path\": [\n",
    "            \"https://learn.microsoft.com/en-us/fabric/get-started/microsoft-fabric-overview\",\n",
    "            \"https://learn.microsoft.com/en-us/fabric/data-science/tuning-automated-machine-learning-visualizations\",\n",
    "        ],\n",
    "        \"chunk_token_size\": 2000,\n",
    "        \"model\": llm_config.config_list[0].model,\n",
    "        \"vector_db\": \"chroma\",  # to use the deprecated `client` parameter, set to None and uncomment the line above\n",
    "        \"overwrite\": True,  # set to True if you want to overwrite an existing collection\n",
    "    },\n",
    "    code_execution_config={\"executor\": code_executor},  # Use the local command line code executor.\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "18",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "#### 2.1 let's ask a question \"List all the Components of Microsoft Fabric\".\n",
    "\n",
    "The answer from **ChatGPT with gpt-4o** at June 7th, 2024 is as below:\n",
    "```\n",
    "Microsoft Fabric is a comprehensive data platform that integrates various services and tools for data management, analytics, and collaboration. As of the latest information available, Microsoft Fabric includes the following components:\n",
    "\n",
    "Data Integration:\n",
    "\n",
    "Azure Data Factory: For creating, scheduling, and orchestrating data workflows.\n",
    "Power Query: A data transformation and data preparation tool.\n",
    "Data Engineering:\n",
    "\n",
    "Azure Synapse Analytics: For big data and data warehousing solutions, including Synapse SQL, Spark, and Data Explorer.\n",
    "Data Science:\n",
    "\n",
    "Azure Machine Learning: For building, training, and deploying machine learning models.\n",
    "Azure Databricks: For collaborative big data and AI solutions.\n",
    "Data Warehousing:\n",
    "\n",
    "...\n",
    "```\n",
    "\n",
    "While the answer from AG2 RAG agent with gpt-4o is as below:\n",
    "```\n",
    "The components of Microsoft Fabric are:\n",
    "\n",
    "1. Power BI\n",
    "2. Data Factory\n",
    "3. Data Activator\n",
    "4. Industry Solutions\n",
    "5. Real-Time Intelligence\n",
    "6. Synapse Data Engineering\n",
    "7. Synapse Data Science\n",
    "8. Synapse Data Warehouse\n",
    "\n",
    "Sources: [Microsoft Fabric Overview](https://learn.microsoft.com/en-us/fabric/get-started/microsoft-fabric-overview)\n",
    "```\n",
    "\n",
    "AG2 RAG agent's answer is exactly the right answer per the official documents while ChatGPT made a few mistakes, it even listed Azure Databricks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "assistant.reset()\n",
    "problem = \"List all the Components of Microsoft Fabric\"\n",
    "chat_result = ragproxyagent.initiate_chat(assistant, message=ragproxyagent.message_generator, problem=problem)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "print(f\"Cost for the chat:\\n{chat_result.cost}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "21",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "#### 2.2 let's ask it to generate AutoML code for us\n",
    "\n",
    "The question is \"Train a regression model, set time budget to 12s, plot the time line plot after training.\".\n",
    "\n",
    "ChatGPT's answer is as below:\n",
    "\n",
    "[It showed a figure]\n",
    "\n",
    "The timeline plot above shows the elapsed time during the training of a linear regression model. The red dashed line indicates the 12-second time budget. The model was trained iteratively, and the plot demonstrates that the training process was monitored to ensure it stayed within the specified time budget.\n",
    "```\n",
    "import time\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets import make_regression\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "# Create a synthetic regression dataset\n",
    "X, y = make_regression(n_samples=1000, n_features=20, noise=0.1)\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
    "\n",
    "# Initialize the model\n",
    "model = LinearRegression()\n",
    "\n",
    "# Record the start time\n",
    "start_time = time.time()\n",
    "\n",
    "# Train the model and record intermediate times\n",
    "times = []\n",
    "time_budget = 12  # in seconds\n",
    "\n",
    "for _ in range(100):\n",
    "    model.fit(X_train, y_train)\n",
    "    current_time = time.time()\n",
    "    elapsed_time = current_time - start_time\n",
    "    times.append(elapsed_time)\n",
    "    if elapsed_time > time_budget:\n",
    "        break\n",
    "\n",
    "# Plot the timeline\n",
    "plt.figure(figsize=(10, 5))\n",
    "plt.plot(times, label='Training time')\n",
    "plt.axhline(y=time_budget, color='r', linestyle='--', label='Time Budget (12s)')\n",
    "plt.xlabel('Iteration')\n",
    "plt.ylabel('Elapsed Time (s)')\n",
    "plt.title('Training Time Line Plot')\n",
    "plt.legend()\n",
    "plt.grid(True)\n",
    "plt.show()\n",
    "```\n",
    "\n",
    "It's not what I need, as ChatGPT has no context of the [AutoML](https://learn.microsoft.com/en-us/fabric/data-science/tuning-automated-machine-learning-visualizations) solution in Fabric Data Science.\n",
    "\n",
    "AG2 RAG agent's answer is much better and ready for deployment. It retrieved the document related to the question and generated code based on the document. It automatically ran the code,  fixed the errors in the code based on the output, and finally it got the correct code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "assistant.reset()\n",
    "problem = \"Train a regression model, set time budget to 12s, plot the time line plot after training.\"\n",
    "\n",
    "chat_result = ragproxyagent.initiate_chat(assistant, message=ragproxyagent.message_generator, problem=problem)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "print(f\"Cost for the chat:\\n{chat_result.cost}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "24",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "Below is the code generated by AG2 RAG agent. It's not a copy of the code in the related document as we asked for different task and training time, but AG2 RAG agent adapted it very well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "import flaml.visualization as fviz\n",
    "from flaml import AutoML\n",
    "from sklearn.datasets import fetch_california_housing\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Load the California housing data and split it into train and test sets\n",
    "housing = fetch_california_housing()\n",
    "x, y = housing.data, housing.target\n",
    "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=7654321)\n",
    "\n",
    "# Create an AutoML instance and set the parameters\n",
    "automl = AutoML()\n",
    "automl_settings = {\n",
    "    \"time_budget\": 12,  # Time limit in seconds\n",
    "    \"task\": \"regression\",  # Type of machine learning task\n",
    "    \"log_file_name\": \"aml_california.log\",  # Name of the log file\n",
    "    \"metric\": \"rmse\",  # Evaluation metric\n",
    "    \"log_type\": \"all\",  # Level of logging\n",
    "}\n",
    "\n",
    "# Fit the AutoML instance on the training data\n",
    "automl.fit(X_train=x_train, y_train=y_train, **automl_settings)\n",
    "\n",
    "# Plot the timeline plot\n",
    "fig = fviz.plot_timeline(automl)\n",
    "fig.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "26",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "### Example 3\n",
    "How to use `MultimodalConversableAgent` to chat with images.\n",
    "\n",
    "Check out this [blog](https://docs.ag2.ai/latest/docs/blog/2023/11/06/LMM-Agent) for more details."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "27",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "We'll ask a question about below image:![image-alt-text](https://th.bing.com/th/id/R.422068ce8af4e15b0634fe2540adea7a?rik=y4OcXBE%2fqutDOw&pid=ImgRaw&r=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "from autogen.agentchat.contrib.multimodal_conversable_agent import MultimodalConversableAgent\n",
    "\n",
    "llm_config = autogen.LLMConfig(\n",
    "    {\n",
    "        \"model\": \"gpt-4o\",\n",
    "        \"http_client\": http_client,\n",
    "        \"api_version\": \"2024-02-01\",\n",
    "        \"api_type\": \"azure\",\n",
    "    },\n",
    "    temperature=0.5,\n",
    "    max_tokens=300,\n",
    ")\n",
    "\n",
    "image_agent = MultimodalConversableAgent(\n",
    "    name=\"image-explainer\",\n",
    "    max_consecutive_auto_reply=10,\n",
    "    llm_config=llm_config,\n",
    ")\n",
    "\n",
    "user_proxy = autogen.UserProxyAgent(\n",
    "    name=\"User_proxy\",\n",
    "    system_message=\"A human admin.\",\n",
    "    human_input_mode=\"NEVER\",  # Try between ALWAYS or NEVER\n",
    "    max_consecutive_auto_reply=0,\n",
    "    code_execution_config={\n",
    "        \"use_docker\": False\n",
    "    },  # Please set use_docker=True if docker is available to run the generated code. Using docker is safer than running the generated code directly.\n",
    ")\n",
    "\n",
    "# Ask the question with an image\n",
    "chat_result = user_proxy.initiate_chat(\n",
    "    image_agent,\n",
    "    message=\"\"\"What's the breed of this dog?\n",
    "<img https://th.bing.com/th/id/R.422068ce8af4e15b0634fe2540adea7a?rik=y4OcXBE%2fqutDOw&pid=ImgRaw&r=0>.\"\"\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "print(f\"Cost for the chat:\\n{chat_result.cost}\")"
   ]
  }
 ],
 "metadata": {
  "front_matter": {
   "description": "Use AG2 in Microsoft Fabric",
   "tags": [
    "agents",
    "chat",
    "microsoft",
    "fabric",
    "guides"
   ]
  },
  "kernel_info": {
   "name": "synapse_pyspark"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  },
  "nteract": {
   "version": "nteract-front-end@1.0.0"
  },
  "spark_compute": {
   "compute_id": "/trident/default"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
